Mixed reality filtering

ABSTRACT

Embodiments that relate to selectively filtering geo-located data items in a mixed reality environment are disclosed. For example, in one disclosed embodiment a mixed reality filtering program receives a plurality of geo-located data items and selectively filtering the data items based on one or more modes. The modes comprise one or more of a social mode, a popular mode, a recent mode, a work mode, a play mode, and a user interest mode. Such filtering yields a filtered collection of the geo-located data items. The filtered collection of data items is then provided to a mixed reality display program for display by a display device.

BACKGROUND

Augmented or mixed reality devices may be used in a variety ofreal-world environments and contexts. Such devices may provide a userwith a real-time view of the physical world surrounding the user, andmay augment the view with holographic objects and other virtual realityinformation.

Large amounts of virtual reality information may be available forpresentation to a user. Some of this information may be associated witha particular location that may be within view of the user. With so muchvirtual reality information available, managing the presentation of thisinformation to a user, and the user's interaction with such information,can prove challenging. Presenting too much virtual reality informationmay clutter a user's experience of a mixed reality environment andoverwhelm the user, making the information difficult to process.Additionally, in some cases the user may be interested in viewing only asmall portion of the total amount of virtual reality informationavailable. Further, even when a smaller portion of the available virtualreality information is presented, the amount of information and mannerof presentation may still create a user experience that is less thandesirable.

SUMMARY

Various embodiments are disclosed herein that relate to presenting afiltered collection of geo-located data items in a mixed realityenvironment via a display device. For example, one disclosed embodimentprovides a method for selectively filtering a plurality of geo-locateddata items that include geo-located hologram items and other geo-locateddata items in a mixed reality environment. The method includes receivingthe plurality of geo-located data items and selectively filtering thedata items based on one or more modes. The modes comprise one or more ofa social mode, a popular mode, a recent mode, a work mode, a play mode,and a user interest mode. Such filtering yields a filtered collection ofthe geo-located data items. The filtered collection of data items isthen provided to a mixed reality display program for display by adisplay device.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view of a mixed reality filtering system accordingto an embodiment of the present disclosure.

FIG. 2 shows an example head-mounted display device according to anembodiment of the present disclosure.

FIG. 3 is a schematic illustration of two users wearing the head-mounteddisplay device of FIG. 2 and viewing an example mixed realityenvironment according to an embodiment of the present disclosure.

FIG. 4 is a schematic view of a starfield depicting a data item densityas seen through a head-mounted display device.

FIG. 5 is a schematic view of a data item starfield displayed within themixed reality environment of FIG. 3 as viewed by a user through thehead-mounted display device.

FIGS. 6A and 6B are a flow chart of a method for selectively filtering aplurality of geo-located data items according to an embodiment of thepresent disclosure.

FIG. 7 is a simplified schematic illustration of an embodiment of acomputing device.

DETAILED DESCRIPTION

FIG. 1 shows a schematic view of one embodiment of a mixed realityfiltering system 10. The mixed reality filtering system 10 includes amixed reality filtering program 14 that may be stored in mass storage 18of a computing device 22. The mixed reality filtering program 14 may beloaded into memory 26 and executed by a processor 30 of the computingdevice 22 to perform one or more of the methods and processes describedin more detail below.

The mixed reality filtering system 10 includes a mixed reality displayprogram 32 that may generate a virtual environment 34 for display on adisplay device, such as the head-mounted display (HMD) device 36, tocreate a mixed reality environment 38. The virtual environment 34includes a plurality of geo-located data items 40. Such data items 40may include one or more virtual images, such as geo-located,three-dimensional hologram items 42 and other geo-located data items 44,such as geo-located two-dimensional virtual objects.

In one example, and as described in more detail below, one or moregeo-located data items 40′ may be received by the computing device 22via a network 46 from a geo-located content channel 48. In anotherexample, one or more geo-located data items 40″ may be generated by anauthoring application 50 located on an authority server 52. Suchgeo-located data item 42″ may also be received by the computing device22 via network 46.

The computing device 22 may take the form of a desktop computing device,a mobile computing device such as a smart phone, laptop, notebook ortablet computer, network computer, home entertainment computer,interactive television, gaming system, or other suitable type ofcomputing device. Additional details regarding the components andcomputing aspects of the computing device 22 are described in moredetail below with reference to FIG. 7.

The computing device 22 may be operatively connected with the HMD device36 using a wired connection, or may employ a wireless connection viaWiFi, Bluetooth, or any other suitable wireless communication protocol.Additionally, the example illustrated in FIG. 1 shows the computingdevice 22 as a separate component from the HMD device 36. It will beappreciated that in other examples the computing device 22 may beintegrated into the HMD device 36.

With reference now also to FIG. 2, one example of an HMD device 200 inthe form of a pair of wearable glasses with a transparent display 54 isprovided. It will be appreciated that in other examples, the HMD device200 may take other suitable forms in which a transparent,semi-transparent or non-transparent display is supported in front of aviewer's eye or eyes. It will also be appreciated that the HMD device 36shown in FIG. 1 may take the form of the HMD device 200, as described inmore detail below, or any other suitable HMD device. Additionally, manyother types and configurations of display devices having various formfactors may also be used within the scope of the present disclosure.Such display devices may include, but are not limited to, hand-heldsmart phones, tablet computers, and other suitable display devices.

With reference to FIGS. 1 and 2, in this example the HMD device 36includes a display system 56 and transparent display 54 that enablesimages such as holographic objects to be delivered to the eyes of a user62. The transparent display 54 may be configured to visually augment anappearance of a physical environment to a user 62 viewing the physicalenvironment through the transparent display. For example, the appearanceof the physical environment may be augmented by graphical content (e.g.,one or more pixels each having a respective color and brightness) thatis presented via the transparent display 54 to create a mixed realityenvironment 38.

The transparent display 54 may also be configured to enable a user toview a physical, real-world object in the physical environment throughone or more partially transparent pixels that are displaying a virtualobject representation. In one example, the transparent display 54 mayinclude image-producing elements located within lenses 204 (such as, forexample, a see-through Organic Light-Emitting Diode (OLED) display). Asanother example, the transparent display 54 may include a lightmodulator on an edge of the lenses 204. In this example the lenses 204may serve as a light guide for delivering light from the light modulatorto the eyes of a user. Such a light guide may enable a user to perceivea 3D holographic image located within the physical environment that theuser is viewing, while also allowing the user to view physical objectsin the physical environment.

The HMD device 36 may also include various sensors and related systems.For example, the HMD device 36 may include an eye-tracking sensor system58 that utilizes at least one inward facing sensor 216. The inwardfacing sensor 216 may be an image sensor that is configured to acquireimage data in the form of eye-tracking information from a user's eyes.Provided the user has consented to the acquisition and use of thisinformation, the eye-tracking sensor system 58 may use this informationto track a position and/or movement of the user's eyes.

The HMD device 36 may also include sensor systems that receive physicalenvironment data from the physical environment. For example, the HMDdevice 36 may include an optical sensor system 60 that utilizes at leastone outward facing sensor 212, such as an optical sensor. Outward facingsensor 212 may detect movements within its field of view, such asgesture-based inputs or other movements performed by a user 62 or by aperson or physical object within the field of view. Outward facingsensor 212 may also capture two-dimensional image information and depthinformation from a physical environment and physical objects within theenvironment. For example, outward facing sensor 212 may include a depthcamera, a visible light camera, an infrared light camera, and/or aposition tracking camera.

The HMD device 36 may include depth sensing via one or more depthcameras. In one example, each depth camera may include left and rightcameras of a stereoscopic vision system. Time-resolved images from oneor more of these depth cameras may be registered to each other and/or toimages from another optical sensor such as a visible spectrum camera,and may be combined to yield depth-resolved video.

In other examples a structured light depth camera may be configured toproject a structured infrared illumination, and to image theillumination reflected from a scene onto which the illumination isprojected. A depth map of the scene may be constructed based on spacingsbetween adjacent features in the various regions of an imaged scene. Instill other examples, a depth camera may take the form of atime-of-flight depth camera configured to project a pulsed infraredillumination onto a scene and detect the illumination reflected from thescene. It will be appreciated that any other suitable depth camera maybe used within the scope of the present disclosure.

Outward facing sensor 212 may capture images of the physical environmentin which a user 62 is situated. In one example, the mixed realitydisplay program 32 may include a 3D modeling system that uses such inputto generate the virtual environment 34 that models the physicalenvironment surrounding the user.

The HMD device 36 may also include a position sensor system 64 thatutilizes one or more motion sensors 224 to enable position trackingand/or orientation sensing of the HMD device. For example, the positionsensor system 64 may be utilized to determine a head pose orientation ofa user's head. In one example, position sensor system 64 may comprise aninertial measurement unit configured as a six-axis or six-degree offreedom position sensor system. This example position sensor system may,for example, include three accelerometers and three gyroscopes toindicate or measure a change in location of the HMD device 36 withinthree-dimensional space along three orthogonal axes (e.g., x, y, z), anda change in an orientation of the HMD device about the three orthogonalaxes (e.g., roll, pitch, yaw).

Position sensor system 64 may also support other suitable positioningtechniques, such as GPS or other global navigation systems. Further,while specific examples of position sensor systems have been described,it will be appreciated that other suitable position sensor systems maybe used.

In some examples, motion sensors 224 may also be employed as user inputdevices, such that a user may interact with the HMD device 36 viagestures of the neck and head, or even of the body. The HMD device 36may also include a microphone system 66 that includes one or moremicrophones 220. In other examples, audio may be presented to the uservia one or more speakers 228 on the HMD device 36.

The HMD device 36 may also include a processor 230 having a logicsubsystem and a storage subsystem, as discussed in more detail belowwith respect to FIG. 7, that are in communication with the varioussensors and systems of the HMD device. In one example, the storagesubsystem may include instructions that are executable by the logicsubsystem to receive signal inputs from the sensors and forward suchinputs to computing device 22 (in unprocessed or processed form), and topresent images to a user via the transparent display 54.

It will be appreciated that the HMD device 36 and related sensors andother components described above and illustrated in FIGS. 1 and 2 areprovided by way of example. These examples are not intended to belimiting in any manner, as any other suitable sensors, components,and/or combination of sensors and components may be utilized. Thereforeit is to be understood that the HMD device 36 may include additionaland/or alternative sensors, cameras, microphones, input devices, outputdevices, etc. without departing from the scope of this disclosure.Further, the physical configuration of the HMD device 36 and its varioussensors and subcomponents may take a variety of different forms withoutdeparting from the scope of this disclosure.

With reference now to FIGS. 3, 4 and 5, descriptions of example usecases and embodiments of the mixed reality filtering system 10 will nowbe provided. FIG. 3 is a schematic illustration of a user 62 wearing HMDdevice 36 and viewing through the device a physical environment 302 inthe form of a city intersection. In this example the HMD device 36 takesthe form of HMD device 200.

As viewed by the user 62, and with reference also to FIG. 1, the virtualenvironment 34 combines with the physical environment 302 to create themixed reality environment 38. The mixed reality environment 38 includesgeo-located data items 40 that are associated with particular locationswithin the physical environment 302. In this example and with referenceto FIG. 3, such geo-located data items 40 include a holographic wizard306, recommendation icons 310 and 314, a social network post 318, acustomer review 322, a photo 326, a popularity icon 330, and an alerticon 334.

As explained in more detail below, numerous additional geo-located dataitems 40 may also be associated with locations within physicalenvironment 302. In various examples such additional geo-located dataitems 40 may number in the tens, hundreds, thousands or more. Where alarge number of additional geo-located data items 40 are present, itwill be appreciated that displaying all such data items may overwhelmthe user 62, make the information difficult to process, and/or createundesirable visual clutter that degrades the user's perception of thephysical environment 302. Advantageously, and as explained in moredetail below, the mixed reality filtering system 10 of the presentdisclosure creates a filtered collection of geo-located data items thatenables a user to easily locate and navigate among data items relevantto the user, thereby providing an improved user experience.

In one example, the mixed reality filtering program 14 is configured toselectively filter a plurality of geo-located data items 40 based on oneor more modes 72 to yield a filtered collection of the geo-located dataitems 68. The filtered collection 68 may then be provided to the mixedreality display program 32 for display by the HMD device 36. In oneexample, the mixed reality filtering program 14 may utilize a socialmode that filters the plurality of geo-located data items 40 to yieldthose data items that are associated with a friend, family member, orother social acquaintance of the user 62.

With reference to FIG. 3, in one example a first friend 370 of the user62 may have previously visited the lounge 350 and left recommendationicon 314 associated with the lounge to indicate the friend's approval ofthe lounge. A second friend 374 of the user 62 may have lunched at thefood cart 354 and left recommendation icon 310 to indicate this friend'sendorsement of the food cart. While in the social mode, in this examplethe mixed reality filtering program 14 may include only therecommendation icons 310 and 314 in the filtered collection of dataitems 68 that are displayed to the user 62 via the HMD device 36.

Additionally and as shown in FIG. 3, each recommendation icon 310 and314 may also take a different form that associates the icon with thefirst friend 370 and the second friend 374, respectively. It will beappreciated that many other types of data items that are associated witha friend, family member, or other social acquaintance of the user mayalso be included in this filtered collection of data items 68.

In another example, the mixed reality filtering program 14 may utilize apopular mode that filters the plurality of geo-located data items 40 toyield those data items that are associated with at least a thresholdpopularity level. With reference again to FIG. 3, the theatre 358 may behosting a musical that is receiving rave reviews from critics in thelocal media, attendees' blogs, commentary via social media and otheronline sources, with such reviews averaging 3.9 stars out of 4 stars. Inone example, the threshold popularity level may correspond to an averagereview of 3 stars out of 4 stars. As the reviews for the musical average3.9 stars, while in the popular mode the mixed reality filtering program14 may include the popularity icon 330 in the filtered collection ofdata items 68. The mixed reality filtering program 14 may display thepopularity icon 330 above the theatre 358 to indicate that a popularevent is associated with the theatre. It will be appreciated that manyother types of data items that are associated with at least a thresholdpopularity level may also be included in this filtered collection ofdata items 68.

In another example, the mixed reality filtering program 14 may utilize arecent mode that filters the plurality of geo-located data items 40 toyield those data items that have been created or updated within athreshold time frame. With reference again to FIG. 3, the customerreview 322 of the lounge 350 may have been posted just one hour ago. Inone example, the threshold timeframe may be within the last week. As thecustomer review 322 of the lounge 350 was posted one hour ago, while inthe recent mode the mixed reality filtering program 14 may include thecustomer review 322 in the filtered collection of data items 68, and maydisplay the review next to the lounge 350. It will be appreciated thatmany other types of data items that have been created or updated withina threshold time frame may also be included in this filtered collectionof data items 68.

In another example, the mixed reality filtering program 14 may utilize awork mode that filters the plurality of geo-located data items 40 toyield those data items that are relevant to the user's work life. Withreference again to FIG. 3, the user 62 may work in commercial realestate, and a colleague of the user may have posted the social networkpost 318 regarding a potential leasing opportunity at that particularcity intersection. While in the work mode the mixed reality filteringprogram 14 may include the social network post 318 in the filteredcollection of data items 68, and may display the post at the streetaddress corresponding to the potential leasing opportunity. It will beappreciated that many other types of data items that are relevant to theuser's work life may also be included in this filtered collection ofdata items 68.

In another example, the mixed reality filtering program 14 may utilize aplay mode that filters the plurality of geo-located data items 40 toyield those data items that relate to leisure, entertainment and/orother recreational activities. With reference again to FIG. 3, thelounge 350 may host regular swing dance events. An attendee of a swingdance event at the lounge 350 may have posted a photograph 326 of peopledancing at the event. While in the play mode the mixed reality filteringprogram 14 may include the photograph 326 in the filtered collection ofdata items 68, and may display the photograph next to the lounge 350. Itwill be appreciated that many other types of data items that relate toleisure, entertainment and/or other recreational activities may also beincluded in this filtered collection of data items 68.

In another example, the mixed reality filtering program 14 may utilize auser interest mode that filters the plurality of geo-located data items40 to yield those data items that relate to one or more interests of theuser. With reference again to FIG. 3, the user 62 may appreciate andenjoy holographic art. In one example, the mixed reality filteringprogram 14 may discern this interest from the user's use of aholographic object creation program and the user's frequent onlinecommentary regarding holographic art. While in the user interest modethe mixed reality filtering program 14 may include the holographicwizard 306 in the filtered collection of data items 68, and may displaythe holographic wizard 306 at a location designated by its creator. Itwill be appreciated that many other types of data items that relate toone or more interests of the user may also be included in this filteredcollection of data items.

It will be appreciated that the user 62 may select one or more of theabove-described modes for use by the mixed reality filtering program 14as described above. It will also be appreciated that other modes inaddition to those describe above may be used by the mixed realityfiltering program 14 to selectively filter the plurality of geo-locateddata items.

The mixed reality filtering program 14 may also prioritize the filteredcollection of data items 68 based on one or more priority factors 74.Once prioritized, the mixed reality filtering program 14 may provide apredetermined number of the prioritized geo-located data items to themixed reality display program 32 for display via the HMD device 36.

In one example an item age priority factor may be used to prioritizedata items according to the most recent data items. For example, wheremultiple reviews of the lounge 350 are available, only the most recentreview 322 of the lounge 350 may be displayed via the HMD device 36. Inother examples, the predetermined number of prioritized geo-locatedreviews may be 2, 3, 4, 5 or more, and such predetermined number ofreviews may be displayed.

In another example an item rating priority factor may be used toprioritize data items according to those items receiving the mostfavorable ratings from commercial reviews, social graph recommendations,etc. For example, the holographic wizard 306 may have received ratingsfrom holographic art critics and friends of the user 62 that average 7.5stars out of 10 stars. Other holographic art creations near the user 62may have average ratings less than 7.5 stars. Accordingly, where onlythe highest rated holographic art creation is to be displayed, only theholographic wizard 306 may be displayed via the HMD device 36. In otherexamples, the predetermined number of prioritized geo-locatedholographic art creations may be 2, 3, 4, 5 or more, and such number ofholographic art creations may be displayed.

In another example user preference data may be used to prioritize dataitems. For example, by examining the user's history of creatingholographic fantasy characters and indicating the user's approval(“liking”) of other holographic fantasy characters, the mixed realityfiltering program 14 may determine that the user has a preference forholographic fantasy characters. Accordingly, the mixed reality filteringprogram may prioritize the display of the holographic wizard 306 over aholographic abstract art creation that is also geo-located near the user62.

In another example item popularity may be used to prioritize data items.For example, the popularity of a data item may correspond to the numberof different comments, reviews, indications of approval, etc. receivedby the data item. With respect to FIG. 3, the holographic wizard 306 mayhave received 550 comments, reviews and indications of approval. Aholographic abstract art creation may have received 50 comments, reviewsand indications of approval. Accordingly, the mixed reality filteringprogram may prioritize the display of the holographic wizard 306 over aholographic abstract art creation based on the higher popularity of thewizard as compared to the abstract art creation.

In another example item source authority may be used to prioritize dataitems. With reference to FIG. 1, in one example a local governmentagency may operate an authority server 52 that includes an authoringapplication 50 that may create and serve geo-located data items 40″ tothe mixed reality filtering program 14. With reference now to FIG. 3,road construction may have closed one lane of the street 362 two blocksfrom the intersection. An alert icon 334 may be generated andgeo-located at the intersection to warn drivers and pedestrians of thelane closure down the street. The local government agency may haveauthority to display geo-located data items via the HMD device 36, andthe mixed reality filtering program 14 may accord such items priorityover other data items. Accordingly, the mixed reality filtering programmay prioritize the display of the alert icon 334 based on the authorityof the local government agency.

In another example commercial precedence may be used to prioritize dataitems. In one example the lounge 350 may pay $1000 to purchase aholographic advertisement 366 promoting a dance contest at the lounge.The food cart 354 may pay $25 to purchase a holographic advertisement(not shown) promoting the food cart. As the lounge 350 paid more for itsadvertisement than the food cart 354, the dance contest advertisement366 may be displayed via the HMD device 36 while the food cartadvertisement may not be displayed.

In another example physical proximity to the user 62 may be used toprioritize data items. In one example the user 62 may specify a prioritydistance from the user within which geo-located data items areprioritized over geo-located data items located beyond such distance.For example, where the user 62 specifies a priority distance of 10 feet,holographic characters from an interactive game being played by a firstfriend 370 standing 20 feet away and across the street from the user maynot be displayed via the HMD device 36. When the user 62 walks acrossthe street to within 10 feet of the first friend 370, the holographiccharacters from the interactive game will now be prioritized anddisplayed.

In another example social proximity to the user 62 may be used toprioritize data items. In one example the user 62 may have a socialgraph that includes the first friend 370 with whom the user 62 interactsvia social networking at least once per day on average. The social graphof the user 62 may also include a second friend 374 with whom the user62 interacts via social networking once per month on average.Accordingly, the mixed reality filtering program 14 may prioritize thegeo-located data items generated by the first friend 370 over thosecreated by the second friend 374 based on the user's more frequentinteractions with the first friend.

The mixed reality filtering program 14 may also be configured to enableand disable one or more authoring applications that are configured toprovide one or more geo-located data items to the mixed reality displayprogram 32 for display by the HMD device 36. One or more authoringapplications 76 may be resident in the mass storage 18 of the computingdevice 22, or may be external to the computing device, such as theauthoring application 50 located on the authority server 52 illustratedin FIG. 1.

In one example and to minimize possible distractions, the mixed realityfiltering program 14 may disable the authoring application 76 when theuser 62 approaches a busy street with many cars driving past the user.In another example, the authority server 52 hosting the authoringapplication 50 may be operated by an airport security office. When theuser 62 enters the airport and crosses into a geo-fenced boundarydefined by the interior of the airport building, the mixed realitydisplay program 32 may enable the authoring application 50 to providegeo-located data items to the mixed reality display program 32 fordisplay via the HMD device 36.

The mixed reality filtering program 14 may also be configured toselectively filter geo-located data items based on one or more datatypes 78 to yield the filtered collection of geo-located data items 68.Data types 78 may include, but are not limited to, social networkingposts, reviews and recommendations, photographs, advertisements, andartwork. For example, the user 62 may desire to see only geo-locateddata items corresponding to social networking posts. Accordingly, theuser may select the social networking post data type, and the mixedreality filtering program 14 may correspondingly filter all geo-locateddata items 40 to yield only social networking posts in the filteredcollection of data items 68 provided to the HMD device 36. It will beappreciated that many other data types of geo-located data items mayalso be used to filter geo-located data items and are within the scopeof the present disclosure.

In another example, the mixed reality filtering program 14 may befurther configured to selectively filter geo-located data items based onone or more third party lenses 80 that apply third party filteringcriteria. For example, the user 62 may admire the art and stylesensibilities of the user's first friend 370. The user's first friend370 may have a First Friend Art & Style lens that applies filteringcriteria to geo-located data items to yield a collection of data itemsthat reflect the first friend's art and style sensibilities.

The user 62 may receive the First Friend Art & Style lens from the firstfriend 370, and this lens may be stored in mass storage 18 of thecomputing device 22. The mixed reality filtering program 14 may thenemploy the First Friend Art & Style lens to selectively filtergeo-located data items to yield a collection of items that reflect thefirst friend's art and style sensibilities. It will be appreciated thatin other examples other third party lenses 80 may be received and/orpurchased by the user 62.

In another example, the mixed reality filtering program 14 may befurther configured to receive the plurality of geo-located data itemsfrom a geo-located content channel 48 when the HMD device 36 is locatedwithin a bounded environment serviced by the geo-located contentchannel. For example, the user 62 and first friend 370 may be sitting inthe lounge 350. The lounge 350 may broadcast a geo-located contentchannel 48 that is accessible when a display device is located withinthe four walls of the lounge. Advantageously, the user 62 and firstfriend 370 may both set their HMD devices 36 to receive and display onlygeo-located data items broadcast by the lounge's geo-located contentchannel 48. Advantageously, in this manner the user 62 and first friend370 can ensure that they are both experiencing a similar mixed realityenvironment, which may enhance their interactions.

In another example, the user 62 and first friend 370 may create anad-hoc geo-located content channel that enables the user and firstfriend to share geo-located data items in a common mixed realityexperience. In this manner, the user 62 and first friend 370 may alsoensure that they are both experiencing a similar mixed realityenvironment, which may enhance their interactions.

In another example, and with reference now to FIGS. 4 and 5, the mixedreality filtering program 14 may be further configured to present thefiltered collection of the geo-located data items 68 in a starfield 400that depicts a data item density in the mixed reality environment 38.The starfield 400 includes a plurality of indicators that eachrepresents a different one of the geo-located data items in the filteredcollection of data items 68. Advantageously, and as schematicallyillustrated in FIG. 4, the starfield 400 provides a user with aconveniently abstracted representation of the density and location ofgeo-located data items within the current view of the user via the HMDdevice 36.

In one example, each of the indicators in the starfield 400 maycorrespond to one of a plurality of data types 78. The mixed realityfiltering program 14 may be further configured to provide each of theplurality of indicators with a data type characteristic that identifiesthe corresponding data type from the plurality of data types. Forexample, the data type characteristic may comprise a particular shape ofthe corresponding indicator. With reference now to FIG. 4, indicators404 having a triangular or pyramid shape may correspond to a socialnetworking post data type. Indicators 408 having a hexagon or hexahedronshape may correspond to a review or recommendation data type. Indicators412 having a circular or spherical shape may correspond to a photographdata type.

FIG. 5 schematically illustrates a data item starfield as viewed by theuser 62 of FIG. 3 through the HMD device 36. In this example, the dataitem starfield is displayed within the mixed reality environment 38 thatincludes the physical environment 302 of the city intersection. The user62 may have selected a social mode that selectively filters theplurality of geo-located data items 40 to yield those items associatedwith a friend, family member, or other social acquaintance of the user62. Accordingly, various indicators that represent social networkingposts, reviews or recommendations, and photographs associated withsocial acquaintances of the user 62 are included within the starfield.

For example, the starfield may include the recommendation icon 310 leftby the second friend 374 next to the food cart 354. In one example, theuser 62 may select the recommendation icon 310 via interaction with theHMD device 36. Upon selection, the icon 310 may expand to display thefull written recommendation of the second friend 374 associated with theicon.

FIGS. 6A and 6B illustrate a flow chart of a method 600 for selectivelyfiltering a plurality of geo-located data items according to anembodiment of the present disclosure. The following description ofmethod 600 is provided with reference to the software and hardwarecomponents of the mixed reality filtering system 10 described above andshown in FIGS. 1-5. It will be appreciated that method 600 may also beperformed in other contexts using other suitable hardware and softwarecomponents.

With reference to FIG. 6A, at 604 the method 600 includes receiving aplurality of geo-located data items. At 608 the geo-located data itemsmay be received from a geo-located content channel when the HMD device36 is located within a bounded environment that is serviced by thegeo-located content channel. At 612 the method 600 includes selectivelyfiltering the plurality of geo-located data items based on one or moremodes to yield a filtered collection of the geo-located data items. Theone or more modes may comprise one or more of a social mode, a popularmode, a recent mode, a work mode, a play mode, and a user interest mode.

At 616 the method 600 includes prioritizing the filtered collection ofthe geo-located data items based on one or more priority factors. At 620the priority factors may include an item age, item ratings, userpreference, item popularity, item source authority, commercialprecedence, physical proximity, and social proximity. At 624 the method600 includes limiting the filtered collection of the geo-located dataitems to a predetermined number of the data items based on the priorityof the data items.

At 628 the method 600 includes selectively filtering the plurality ofgeo-located data items based on one or more data types to yield thefiltered collection of the geo-located data items. At 632 the method 600includes selectively filtering the plurality of geo-located data itemsbased on one or more third party lenses that apply third party filteringcriteria. At 636 the method 600 includes providing the filteredcollection of the geo-located data items 68 to the mixed reality displayprogram 32 for display by the HMD device 36.

At 640 the method 600 includes enabling one or more authoringapplications that are configured to provide one or more of thegeo-located data items to the mixed reality display program for displayby the display device. With reference now to FIG. 6B, at 644 the method600 includes presenting the filtered collection of the geo-located dataitems in a starfield that depicts a data item density in the mixedreality environment, with the starfield comprising a plurality ofindicators that each represents a different one of the geo-located dataitems in the filtered collection. At 648 each of the geo-located dataitems corresponds to one of a plurality of data types, and the method600 further includes providing each of the plurality of indicators witha data type characteristic that identifies the corresponding data typefrom the plurality of data types.

It will be appreciated that method 600 is provided by way of example andis not meant to be limiting. Therefore, it is to be understood thatmethod 600 may include additional and/or alternative steps than thoseillustrated in FIGS. 6A and 6B. Further, it is to be understood thatmethod 600 may be performed in any suitable order. Further still, it isto be understood that one or more steps may be omitted from method 600without departing from the scope of this disclosure.

FIG. 7 schematically shows a nonlimiting embodiment of a computingsystem 700 that may perform one or more of the above described methodsand processes. Computing device 22 may take the form of computing system700. Computing system 700 is shown in simplified form. It is to beunderstood that virtually any computer architecture may be used withoutdeparting from the scope of this disclosure. In different embodiments,computing system 700 may take the form of a mainframe computer, servercomputer, desktop computer, laptop computer, tablet computer, homeentertainment computer, network computing device, mobile computingdevice, mobile communication device, gaming device, etc. As noted above,in some examples the computing system 700 may be integrated into an HMDdevice.

As shown in FIG. 7, computing system 700 includes a logic subsystem 704and a storage subsystem 708. Computing system 700 may optionally includea display subsystem 712, a communication subsystem 716, a sensorsubsystem 720, an input subsystem 722 and/or other subsystems andcomponents not shown in FIG. 7. Computing system 700 may also includecomputer readable media, with the computer readable media includingcomputer readable storage media and computer readable communicationmedia. Computing system 700 may also optionally include other user inputdevices such as keyboards, mice, game controllers, and/or touch screens,for example. Further, in some embodiments the methods and processesdescribed herein may be implemented as a computer application, computerservice, computer API, computer library, and/or other computer programproduct in a computing system that includes one or more computers.

Logic subsystem 704 may include one or more physical devices configuredto execute one or more instructions. For example, the logic subsystem704 may be configured to execute one or more instructions that are partof one or more applications, services, programs, routines, libraries,objects, components, data structures, or other logical constructs. Suchinstructions may be implemented to perform a task, implement a datatype, transform the state of one or more devices, or otherwise arrive ata desired result.

The logic subsystem 704 may include one or more processors that areconfigured to execute software instructions. Additionally oralternatively, the logic subsystem may include one or more hardware orfirmware logic machines configured to execute hardware or firmwareinstructions. Processors of the logic subsystem may be single core ormulticore, and the programs executed thereon may be configured forparallel or distributed processing. The logic subsystem may optionallyinclude individual components that are distributed throughout two ormore devices, which may be remotely located and/or configured forcoordinated processing. One or more aspects of the logic subsystem maybe virtualized and executed by remotely accessible networked computingdevices configured in a cloud computing configuration.

Storage subsystem 708 may include one or more physical, persistentdevices configured to hold data and/or instructions executable by thelogic subsystem 704 to implement the herein described methods andprocesses. When such methods and processes are implemented, the state ofstorage subsystem 708 may be transformed (e.g., to hold different data).

Storage subsystem 708 may include removable media and/or built-indevices. Storage subsystem 708 may include optical memory devices (e.g.,CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory devices(e.g., RAM, EPROM, EEPROM, etc.) and/or magnetic memory devices (e.g.,hard disk drive, floppy disk drive, tape drive, MRAM, etc.), amongothers. Storage subsystem 708 may include devices with one or more ofthe following characteristics: volatile, nonvolatile, dynamic, static,read/write, read-only, random access, sequential access, locationaddressable, file addressable, and content addressable.

In some embodiments, aspects of logic subsystem 704 and storagesubsystem 708 may be integrated into one or more common devices throughwhich the functionally described herein may be enacted, at least inpart. Such hardware-logic components may include field-programmable gatearrays (FPGAs), program- and application-specific integrated circuits(PASIC/ASICs), program- and application-specific standard products(PSSP/ASSPs), system-on-a-chip (SOC) systems, and complex programmablelogic devices (CPLDs), for example.

FIG. 7 also shows an aspect of the storage subsystem 708 in the form ofremovable computer readable storage media 724, which may be used tostore data and/or instructions executable to implement the methods andprocesses described herein. Removable computer-readable storage media724 may take the form of CDs, DVDs, HD-DVDs, Blu-Ray Discs, EEPROMs,and/or floppy disks, among others.

It is to be appreciated that storage subsystem 708 includes one or morephysical, persistent devices. In contrast, in some embodiments aspectsof the instructions described herein may be propagated in a transitoryfashion by a pure signal (e.g., an electromagnetic signal, an opticalsignal, etc.) that is not held by a physical device for at least afinite duration. Furthermore, data and/or other forms of informationpertaining to the present disclosure may be propagated by a pure signalvia computer-readable communication media.

When included, display subsystem 712 may be used to present a visualrepresentation of data held by storage subsystem 708. As the abovedescribed methods and processes change the data held by the storagesubsystem 708, and thus transform the state of the storage subsystem,the state of the display subsystem 712 may likewise be transformed tovisually represent changes in the underlying data. The display subsystem712 may include one or more display devices utilizing virtually any typeof technology. Such display devices may be combined with logic subsystem704 and/or storage subsystem 708 in a shared enclosure, or such displaydevices may be peripheral display devices. The display subsystem 712 mayinclude, for example, the display system 56 and transparent display 54of the HMD device 36.

When included, communication subsystem 716 may be configured tocommunicatively couple computing system 700 with one or more networksand/or one or more other computing devices. Communication subsystem 716may include wired and/or wireless communication devices compatible withone or more different communication protocols. As nonlimiting examples,the communication subsystem 716 may be configured for communication viaa wireless telephone network, a wireless local area network, a wiredlocal area network, a wireless wide area network, a wired wide areanetwork, etc. In some embodiments, the communication subsystem may allowcomputing system 700 to send and/or receive messages to and/or fromother devices via a network such as the Internet.

Sensor subsystem 720 may include one or more sensors configured to sensedifferent physical phenomenon (e.g., visible light, infrared light,sound, acceleration, orientation, position, etc.) as described above.Sensor subsystem 720 may be configured to provide sensor data to logicsubsystem 704, for example. As described above, such data may includeeye-tracking information, image information, audio information, ambientlighting information, depth information, position information, motioninformation, user location information, and/or any other suitable sensordata that may be used to perform the methods and processes describedabove.

When included, input subsystem 722 may comprise or interface with one ormore sensors or user-input devices such as a game controller, gestureinput detection device, voice recognizer, inertial measurement unit,keyboard, mouse, or touch screen. In some embodiments, the inputsubsystem 722 may comprise or interface with selected natural user input(NUI) componentry. Such componentry may be integrated or peripheral, andthe transduction and/or processing of input actions may be handled on-or off-board. Example NUI componentry may include a microphone forspeech and/or voice recognition; an infrared, color, stereoscopic,and/or depth camera for machine vision and/or gesture recognition; ahead tracker, eye tracker, accelerometer, and/or gyroscope for motiondetection and/or intent recognition; as well as electric-field sensingcomponentry for assessing brain activity.

The term “program” may be used to describe an aspect of the mixedreality filtering system 10 that is implemented to perform one or moreparticular functions. In some cases, such a program may be instantiatedvia logic subsystem 704 executing instructions held by storage subsystem708. It is to be understood that different programs may be instantiatedfrom the same application, service, code block, object, library,routine, API, function, etc. Likewise, the same program may beinstantiated by different applications, services, code blocks, objects,routines, APIs, functions, etc. The term “program” is meant to encompassindividual or groups of executable files, data files, libraries,drivers, scripts, database records, etc.

It is to be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated may beperformed in the sequence illustrated, in other sequences, in parallel,or in some cases omitted. Likewise, the order of the above-describedprocesses may be changed.

The subject matter of the present disclosure includes all novel andnonobvious combinations and subcombinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

1. A mixed reality filtering system for presenting a filtered collectionof geo-located data items in a mixed reality environment, the mixedreality filtering system comprising: a display device operativelyconnected to a computing device, the display device including a displaysystem for presenting holographic objects; and a mixed reality filteringprogram executed by a processor of the computing device, the mixedreality filtering program configured to: receive a plurality ofgeo-located data items comprising geo-located hologram items and othergeo-located data items; selectively filter the plurality of geo-locateddata items based on one or more modes to yield the filtered collectionof geo-located data items, the one or more modes comprising one or moreof a social mode, a popular mode, a recent mode, a work mode, a playmode, and a user interest mode; and provide the filtered collection ofgeo-located data items to a mixed reality display program for display bythe display device.
 2. The mixed reality filtering system of claim 1,wherein the mixed reality filtering program is further configured to:prioritize the filtered collection of geo-located data items based onone or more priority factors; and provide a predetermined number of theprioritized geo-located data items to the mixed reality display programfor display by the display device.
 3. The mixed reality filtering systemof claim 2, wherein the one or more priority factors are selected fromthe group consisting of item age, item ratings, user preference data,item popularity, item source authority, commercial precedence, physicalproximity, and social proximity.
 4. The mixed reality filtering systemof claim 1, wherein the mixed reality filtering program is furtherconfigured to enable and disable one or more authoring applications thatare configured to provide one or more of the geo-located data items tothe mixed reality display program for display by the display device. 5.The mixed reality filtering system of claim 1, wherein the mixed realityfiltering program is further configured to selectively filter theplurality of geo-located data items based on one or more data types toyield the filtered collection of the geo-located data items.
 6. Themixed reality filtering system of claim 1, wherein the mixed realityfiltering program is further configured to selectively filter theplurality of geo-located data items based on one or more third partylenses that apply third party filtering criteria.
 7. The mixed realityfiltering system of claim 1, wherein the mixed reality filtering programis further configured to receive the plurality of geo-located data itemsfrom a geo-located content channel when the display device is locatedwithin a bounded environment serviced by the geo-located contentchannel.
 8. The mixed reality filtering system of claim 1, wherein themixed reality filtering program is further configured to present thefiltered collection of the geo-located data items in a starfield thatdepicts a data item density in the mixed reality environment, thestarfield comprising a plurality of indicators that each represents adifferent one of the geo-located data items in the filtered collection.9. The mixed reality filtering system of claim 8, wherein each of thegeo-located data items corresponds to one of a plurality of data types,and the mixed reality filtering program is further configured to provideeach of the plurality of indicators with a data type characteristic thatidentifies the corresponding data type from the plurality of data types.10. The mixed reality filtering system of claim 1, wherein the displaydevice comprises a head-mounted display device operatively connected tothe computing device.
 11. A method for selectively filtering a pluralityof geo-located data items that include geo-located hologram items andother geo-located data items in a mixed reality environment, comprising:receiving the plurality of geo-located data items; selectively filteringthe plurality of geo-located data items based on one or more modes toyield a filtered collection of the geo-located data items, the one ormore modes comprising one or more of a social mode, a popular mode, arecent mode, a work mode, a play mode, and a user interest mode; andproviding the filtered collection of geo-located data items to a mixedreality display program for display by a display device.
 12. The methodof claim 11, wherein selectively filtering the plurality of geo-locateddata items further comprises: prioritizing the filtered collection ofgeo-located data items based on one or more priority factors; andlimiting the filtered collection of geo-located data items to apredetermined number of the data items based on the priority of the dataitems.
 13. The method of claim 12, wherein the one or more priorityfactors are selected from the group consisting of item age, itemratings, user preference data, item popularity, item source authority,commercial precedence, physical proximity, and social proximity.
 14. Themethod of claim 11, further comprising enabling one or more authoringapplications that are configured to provide one or more of thegeo-located data items to the mixed reality display program for displayby the display device.
 15. The method of claim 11, further comprisingselectively filtering the plurality of geo-located data items based onone or more data types to yield the filtered collection of geo-locateddata items.
 16. The method of claim 11, further comprising selectivelyfiltering the plurality of geo-located data items based on one or morethird party lenses that apply third party filtering criteria.
 17. Themethod of claim 11, further comprising receiving the plurality ofgeo-located data items from a geo-located content channel when thedisplay device is located within a bounded environment serviced by thegeo-located content channel.
 18. The method of claim 11, furthercomprising presenting the filtered collection of the geo-located dataitems in a starfield that depicts a data item density in the mixedreality environment, the starfield comprising a plurality of indicatorsthat each represents a different one of the geo-located data items inthe filtered collection.
 19. The mixed reality filtering system of claim18, wherein each of the geo-located data items corresponds to one of aplurality of data types, and further comprising providing each of theplurality of indicators with a data type characteristic that identifiesthe corresponding data type from the plurality of data types.
 20. Amethod for selectively filtering a plurality of geo-located data itemsthat include geo-located hologram items and other geo-located data itemsin a mixed reality environment, comprising: receiving the plurality ofgeo-located data items; selectively filtering the plurality ofgeo-located data items based on one or more modes to yield a filteredcollection of geo-located data items, the one or more modes comprisingone or more of a social mode, a popular mode, a recent mode, a workmode, a play mode, and a user interest mode; providing the filteredcollection of geo-located data items to a mixed reality display programfor display by a display device, wherein the filtered collection of thegeo-located data items is represented by a starfield that depicts a dataitem density in the mixed reality environment, the starfield comprisinga plurality of indicators that each represents a different one of thegeo-located data items in the filtered collection; and wherein each ofthe geo-located data items corresponds to one of a plurality of datatypes, providing each of the plurality of indicators with a data typecharacteristic that identifies the corresponding data type from theplurality of data types.